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When it comes to reading, technology has changed everything for 
students who are deaf or hard of hearing. Reading was an experience 
that was text based; it consisted of negotiating combinations of words 
expressed in linear form on paper. How was it possible to develop 
reading materials for bilingual readers, especially if one of the 
youngsters’ languages was visual and spatial? 

Enter technology. The touchscreen made a revolutionary difference. In an 
unprecedented way, touchscreen tablets complemented our conventional and 
traditional printed books. 

Enter the Visual Language and Visual Learning Lab (VL2). At VL2, the Science of 
Learning Center funded by the National Science Loundation at Gallaudet University, 
researchers explore themes of literacy and bilingualism for deaf children—and 
translate their findings into apps that allow deaf children to learn to read. 

Enter The Baobab. The Baobab , the story of a curious little girl who embarks on a 
search for a fruit growing from a rare tree and gets into mishaps along the way, was 
the first downloadable app developed at VL2. Using the latest technology to provide 
reading for young deaf children, The Baobab opened a new field in education. Lor the 
first time, deaf children could participate equally with their hearing peers in 
exploring literature while developing literacy skills in American Sign Language 
(ASL) and English. 

As we watched children enjoy the app, we studied how they read, how they 
navigated between English and ASL, and how both languages helped them develop 
strong bilingual literacy skills. 

The Benefits of Bilingualism 

In recent years, researchers have turned their attention to the cognitive impact of 
bilingualism, and the benefits of using two languages have become increasingly 
apparent. Children raised in bilingual families exhibit stronger awareness of the style 
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and tone of language, stronger cognitive 
development, and higher levels of reading skill 
than children raised in families where only one 
language is used (Allen, 2015; Kovelman, 
Berens, & Petitto, 2013; Bialystok, Craig, 
Green, & Gollan, 2009; Jasinska & Petitto, 
2013,2014; Petitto, 2009). 

To maximize the bilingual advantage, 
children must be exposed to both languages in 
their early preschool years. Unfortunately many 
deaf children, with tremendous potential for 
becoming ASL/English bilinguals, do not get 
adequate exposure to either language. These 
children miss out on much-needed linguistic 
experience—which has a huge impact on their 
learning and academic success (Petitto, 2009). 

British researcher Gary Morgan (2006) found 
that deaf children rely on and benefit from sign 
language narratives. Narratives in ASL provide 
children with “opportunities to develop 
potential cognitive flexibility and 
metalinguistic abilities in order to facilitate the 
development of English literacy skills” (p. 338). 


In other words, exposure to ASL narratives and 
vocabulary allows children to develop skills to 
grasp concepts related to the understanding of 
English, and this, in turn, helps them become 
skilled readers. 

Hanson and Padden (1990, 1994) found that 
when students read a story through a software 
program called HandsOn, which presents ASL 
and English side by side, they understood the 
story better than when they read the story only 
in printed English. Morgan (2006) concludes 
that this skill in comprehension is due to how 
children engage in a “contrastive narrative 
analysis task” (i.e., how children receive with 
both languages, processing each separately and 
deriving meaning through both). 

It is this use of two languages—and the 
resulting contrastive narrative analysis—that 
stimulates the child’s knowledge of how each 
language works and maximizes his or her ability 
to process linguistic information. Past studies 
have found that deaf children who experience 
stories in ASL and printed English are more 
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stimulated mentally than those who experience stories in only 
one of these languages, and this stimulation facilitates their 
becoming skilled readers (Morgan, 2006). 

With the technological advances of touchscreen computing, 
we can develop bilingual resources to encourage cognitive 
development, bilingualism, and literacy growth in deaf children. 
Adam Stone (2014), who authored an ASL/English e-book, 
noted that both languages could be provided interactively on a 
touchscreen with vivid and sophisticated video capabilities. This 
allows children bilingual access to literature. 

Kennedy (2004) found that middle school deaf students 
demonstrated higher levels of motivation and engagement 
when texts were accompanied by ASL presentations of the 
material. When they clicked on printed words, students 
expressed a preference for representations of the ASL sign 
equivalents. 

Storybook Apps: Exciting in Two Languages 

The Baobab , the first story app presented in ASL/English, was 
soon followed by The Boy Who Cried Wolf , The Blue Lobster , and 
The Solar System. Developed and designed under Melissa 
Malzkuhn, in VL2’s Motion Light Lab, each storybook app 
provides young readers with rich, engaging literature in both 
ASL and English. 

Research has shown that exposure to sign language 
storytelling nurtures the childrens development of vocabulary 
and increases their skills in language and literacy (Berke, 2013; 
Mayberry, del Giudice, & Lieberman, 2011). Each app 
incorporates the latest research in language learning and 
bilingualism from research done at VL2 and established 
principles of literacy development; and each offers three ways of 
interacting with the content, identified as “watch,” “read,” and 
“learn.” 

WATCH—VIEWING THE NARRATIVE 

In the “watch” mode, children view the narrative presented by a 
professional ASL storyteller in conjunction with high-interest 
visuals and animation. The extended narration, without any 
editing or interference, requires 
that young viewers pay attention 
and enjoy the flow, rhythm, and 
other elements of ASL storytelling. 

Encouraging children to watch 
stories in ASL helps them learn to 
follow and understand the 
sequencing of events within a 
narrative and become familiar with 
the structures of narratives. This 
understanding prepares them for 
the process of reading, a process in 
which they will go from the whole 
story to its smaller parts (e.g., 
paragraphs, sentences, words, 
letters). 

4 


READ—ENJOYING THE TEXT 

In the “read” mode, children view pages of English text on 
screen in a manner similar to the way most books are viewed in 
traditional children’s literature. Each page features sentences in 
English accompanied by illustrations that help children connect 
words to meaning. There are also plenty of interactive options. 
Children can check out parts of the story in ASL or view videos 
showing the signs and fingerspelling for English vocabulary. 

At their own choosing, they can alternate between ASL and 
English, tapping an on-screen “play” button that results in a 
pop-up window with an ASL translation of the page. This 
potential page-by-page translation allows children to focus on 
smaller units of meaning than those in the “watch” mode as 
well as to see how the two languages handle expression of 
similar information. 

The “read” mode addresses one of the obstacles that many 
deaf children face as readers—an insufficient knowledge of 
vocabulary (Torgesen, 1986). Without understanding 
vocabulary, it is difficult to develop motivation and 
engagement (Herzig, 2009; 2014.) Motivation is what makes a 
person want to read, and engagement is what makes a person 

keep reading (Guthrie & 
Wigfield, 2000). In the “read” 
mode, children maintain both 
their motivation and 
engagement by proceeding at 
their own pace, choosing either 
to watch the ASL version of 
the page, read the text, or view 
the vocabulary words in ASL as 
signs or as fingerspelled words. 

LEARN—VOCABULARY FOR ALL 

In the “learn” mode, children 
are presented with a glossary 
that has vocabulary from the 
story in an alphabetized list. 
Children tap on each word to 
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see its translation into ASL. 

Translations both in the glossary and in text are handled 
through videos that incorporate “chaining”—a natural feature 
of ASL in which signers present a word in signed ASL, then 
fingerspell the word, then repeat the word in signed ASL. 
Research has demonstrated the usefulness of chaining as a tool 
to enhance literacy development (Humphries & MacDougall, 
1999; Padden & Ramsey, 2000). Chaining helps the child build 
joint internal representations of the word in both languages and 
provides a means for linking fingerspelling to English print. 

The “learn” mode emphasizes vocabulary; children deepen 
the connection to meaning between signed or printed words 
when they return to the story in “read” mode. 

Lesson Plans— 

Complementary and 
Complimentary 

Lesson plans are available for free 
for the storybooks. They can be 
downloaded in PDF from 
www. VL2storybookapps. com. The 
lessons are designed to promote 
ASL skills, awareness of how ASL 
works, and literacy in the 
classroom; they also encourage 
further immersion in reading. 

Creating the Personal 
Storybook 

In an effort to expand the field and 
the availability of bilingual storybook apps, VL2’s Motion 
Light Lab is releasing a program called Storybook Creator, 
which allows an individual to create an app without writing a 
line of code. Users upload text, images, and videos to create 
new storybook apps through Storybook Creator in Xcode, 
Apple’s standard app development software. This is a fantastic 
opportunity for educators, storytellers, artists, designers, and 
parents to create stories that benefit the bilingual learner. 


ASL on iPads: Expanding the Field 

Since the launch of Apple’s iPad in 2010, over 20 ASL/English 
e-books and storybook apps have been created and the demand 
is high. This is an emerging field, and developers and educators 
need more empirical research in app design for bilingual 
readers, how children use the ASL/English readers, and the 
readers’ effectiveness in literacy development. 

At the time of writing, we were conducting assessments on 
the user experience of our storybook apps. We’re looking at 44 
deaf and hard of hearing students from four different schools, 
tracking how students use the app, how they interact with it, 
and to what they pay attention. From what we’ve seen, the 
majority of children, regardless of their language skills and 
hearing levels, show an interest in both languages. 

In the Petitto Brain and Language Laboratory for 
Neuroimaging (BL2), we, with Dr. Laura-Ann Petitto and her 
team of students, many of whom are part of the PhD in 
Educational Neuroscience (PEN) program, are currently 
conducting a NSF-VL2 Science of Learning Center study with 
modern fNIRS brain imaging and eye tracking technology to 
measure precisely how children perceive and process the rich 
moving visual scenes in storybook apps. In this first educational 
neuroscience study of how young children process bilingual 
visual linguistic information—with one language in sign and 
one language in print—we seek to learn whether segmentation 
at the heart of fingerspelling in ASL may facilitate the young 
child’s segmentation of visually presented printed strings of 

English letters, words, and 
sentences on the page en 
route to becoming 
successful readers. We are 
also examining whether 
deaf children who learn 
sign language early in life 
are advantaged or not in the 
visual linguistic processing 
of storybook apps relative 
to deaf children who learn 
sign language later. 

At the same time, we 
continue to develop apps 
for children to enjoy, and 
we build on what the 
research has shown: 
Bilingualism—the mastery of two languages instead of just 
one—benefits all children. Deaf and hard of hearing children 
who are bilingual experience a deeper understanding of 
language, increased skills in reading, literacy, and enhanced 
cognitive development. As we look at our students and how 
they read, and as we develop new apps for them to learn from 
and enjoy, this knowledge is what infuses the purpose of our 
work. 
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